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The  Pacific  Science  Center's  1973 
summer  exhibition,  "Energy  and  Its 
Relation  to  Man  and  the  Environment," 
was  studied  to  determine  the  effec- 
tiveness of  evaluation  techniques. 
The  six  techniques  tested  are  rated 
in  table  2 . 

When  a  presentation  is  being 
developed,  problems  are  frequently 
obvious  and  the  speed  of  evaluation 
is  generally  more  important  than  its 
precision;  otherwise,  personal 


involvement  may  become  so  great  that 
change  is  resisted.  Evaluation 
before  completion  should  be  routine 
so  problems  can  be  corrected  before 
investments  in  time,  energy,  and 
money  become  large. 

With  some  understanding  of  the 
biases  inherent  in  nonrepresentative 
samples  of  visitors,  interpreters 
should  be  able  to  use  inexpensively 
collected  information  to  increase 
the  effectiveness  of  presentations. 
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Introduction 

In  visitor  centers  and  other 
settings  for  interpretation,  evalua- 
tion of  effectiveness  has  been  widely 
recognized  as  important  but  seldom 
accomplished.     Specific  evaluation 
criteria  and  methods  have  not  been 
readily  available,  and  many  staffs 
are  too  burdened  with  other  duties 
to  work  out  evaluation  procedures. 
In  addition,  the  high  costs  of  many 
displays,  exhibits,  and  other 
unmanned  presentations  often  make 
experimentation  impractical. 

A  unique  opportunity  for  evalua- 
tion arose  in  1973  when  the  staff 
of  the  Pacific  Science  Center 
created  a  "summer  special"  exhibition 
on  energy  and  then  modified  it  on 
the  basis  of  evaluation  by  the 
Environmental  Interpretation  Research 
Project  of  the  Pacific  Northwest 
Forest  and  Range  Experiment  Station. 

The  Pacific  Science  Center 
and  the  Energy  Special 

Built  as  the  Science  Pavilion  of 
the  Seattle  World's  Fair,  the  Pacific 
Science  Center  comprises  four  buildings 
and  is  dedicated  to  helping  the  public 
understand  science.     In  addition  to 
its  permanent  exhibits  on  space 
technology,  astronomy,  mathematics, 
and  the  physical  and  life  sciences, 
the  center  has  for  several  years 
offered  "summer  specials."  These 
add  a  dynamic  element  to  the  Center's 
normal  attractions,  creating  topical 
additions  that  encourage  visitors  to 
return . 

The  theme  for  the  1973  summer 
special  was  "Energy  and  Its  Relation 
to  Man  and  the  Environment."  This 
"Energy  Special"  was  set  up  in  the 
Center's  Allen  Building  with  the 
floor  space  of  approximately  80  by 
104  feet   (24  by  31  meters)  divided 
into  five  parts.     Four  parts  were  used 
for  the  presentations  listed  in  table  1. 
The  fifth  was  used  as  a  reading  area. 


Nearly  all  visitors  to  the  Energy 
Special  paid  admissions  of  $1.50  per 
adult  and  $0.50  per  child  under  16. 
This  undoubtedly  affected  audience 
composition . 

Procedures 

Evaluation  began  only  after  the 
Energy  Special  was  complete  enough  to 
open--a  month  and  a  half  after  con- 
struction began.     The  first  evaluation 
data  were  collected  from  an  invited 
audience  on  June  28,  3  days  ahead  of 
the  public  opening.     Evaluation  data 
were  collected  until  the  exhibition 
closed  on  September  1,  1973. 

In  all,  nine  analyses  were  made. 
Six  were  evaluations:    evaluation  by 
a  panel  of  outsiders,  volunteered 
comments,  observed  audience  attention, 
time- lapse  photography,  visitor  voting 
at  individual  presentations,  and 
following  sample  visitors.     The  other 
three  examined  hours  of  attendance, 
visitor  characteristics,  and  interest 
in  possible  themes  for  future  summer 
specials . 

PANEL  OF  OUTSIDERS 

During  the  summer,  six  "outsiders" 
were  invited  to  visit  the  Energy  Special 
and  then  to  be  interviewed  concerning 
what  they  observed.     None  of  the  six 
was  a  specialist  in  interpretation, 
and  none  was  associated  with  the  Pacific 
Science  Center.     However,  all  were 
professional  people  with  some  experience 
or  personal  interest  in  interpreting 
scientific  topics  to  the  public. 

VOLUNTEERED  COMMENTS 

A  suggestion  box  was  installed  in 
the  reading  area  with  a  note  pad  and 
pencil  and  a  sign  reading  "We  appreciate 
your  comments.     Please  leave  us  a  note." 
To  avoid  comments  on  other  parts  of  the 
Pacific  Science  Center,  the  words  "Limit 
comments  to  Energy  Special"  were  later 
added. 
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Comments  received  during  visitor 
voting  (discussed  under  "Visitor 
Voting")  were  combined  with  comments 
from  the  suggestion  box  and  grouped 
into  those  received  between  June  28 
and  July  16  and  those  received 
between  July  19  and  August  15. 

OBSERVED  AUDIENCE  ATTENTION 

Because  presentations  must  hold 
attention  if  visitors  are  to  enjoy 
and  learn  from  them,  audience  atten- 
tion was  observed  at  the  puppet  theater 
and  the  laboratory  area.!/ 

Two  observers  were  located  where 
they  could  unobtrusively  observe  the 
eyes  of  audience  members.     At  2-minute 
intervals,  they  scanned  the  audience 
and  recorded  the  number  of  people 
watching  the  speaker  or  what  he  was 
presenting.     Data  were  converted  into 
percentages  of  people  present  and  then 
graphed . 

TIME-LAPSE  PHOTOGRAPHY 

To  avoid  some  of  the  limitations 
of  directly  observing  visitors, 
time-lapse  movies  were  taken  during 
a  4-day  test.    These  used  a  super-8 
movie  camera  with  zoom  lens  and 
"single  framing"  capability.  Colored 
film  (ASA  speed  160)  was  used  without 
supplemental  lighting.    The  camera 
was  triggered  by  a  cable  release 
connected  to  a  solenoid  activated  by 
microswitches  and  cams  on  an  inexpen- 
sive electric  clock  motor.    When  gear 
ratios  were  altered  exposures  could 
be  obtained  at  intervals  of  4,  5.5, 
8,  16,  or  32  seconds.     The  entire 
assembly  was  mounted  approximately 
10  feet  above  the  floor  and  50  feet 
from  the  area  photographed. 


—    Dick,  Ronald  E. ,  Erik  Myklestad,  and 
J.  Alan  Wagar.     1975.     Audience  attention  as 
a  basis  for  evaluating  interpretive  presenta- 
tions.    USDA  For.  Serv.  Res.  Pap.  PNW-198, 
7  p.,  illus.     Pac.  Northwest  For.  and  Range 
Exp.  Stn. ,  Portland,  Oreg. 


VISITOR  VOTING 

To  permit  visitors  to  vote  directly 
on  individual  presentations,  voting  boxes 
and  packets  were  provided  from  June  28 
through  July  17  and  again  from  August  21 
through  August  31. 

At  each  of  six  presentations,  a 
box  with  five  slotted  compartments  was 
provided,  permitting  visitors  to  vote 
from  1  (poor)  to  5  (good)  on  each 
presentation.     A  colored  photograph  on 
each  box  showed  the  presentation  to  which 
it  applied.     As  the  Energy  Special 
evolved,  some  boxes  were  shifted  to 
sample  newly  completed  presentations. 

Each  voting  packet  included  a  card 
with  six  voting  slips  attached--one 
for  each  presentation.     A  number  on 
each  card  and  its  attached  slips  made 
it  possible  to  determine  how  many  votes 
each  visitor  cast. 

At  randomly  selected  times  each 
day  during  the  two  sample  periods,  voting 
packets  were  given  to  as  many  visitors 
as  possible--with  two  constraints. 
First,  packets  were  given  only  to  visitors 
judged  to  be  12  or  older  and  capable  of 
understanding  instructions;  second,  no 
visitor  received  a  packet  without  a 
complete  explanation  of  its  use. 

Although  each  visitor  received  only 
one  voting  packet,  three  different  packets 
were  used,  each  with  a  different  question. 
Each  visitor  was  therefore  asked  only 
one  of  the  questions:     "How  effectively 
was  the  information  presented?"  "How 
enjoyable  did  you  find  the  subject 
matter?"     and  "How  much  new  information 
did  you  gain?"    To  aid  tabulation  of 
answers,  slips  of  a  different  color  were 
used  for  each  question. 

Slips  were  collected  daily  and 
sorted  by  question  (indicated  by  color 
of  slip)  and  slot  (1  to  5).  The 
"average"  vote  was  then  computed  (on  a 
scale  of  1  to  5)  for  each  question  at 
each  presentation. 


3 


FOLLOWING  SAMPLE  VISITORS 

Between  July  24  and  August  17, 
171  randomly  selected  visitors  were 
followed  unobtrusively  through  the 
Energy  Special.    The  observer  carried 
recording  forms  and  two  stopwatches-- 
one  to  record  the  visitor's  total 
time  in  the  building,  the  other  to 
record  his  time  at  each  exhibit  or 
demonstration. 

Because  most  presentations 
required  at  least  some  time  for 
understanding,  a  "minimum  learning 
time"  was  defined  for  each,  based 
on  the  judgment  of  demonstration 
personnel  and  managers  familiar  with 
both  visitor  behavior  and  the  intended 
message  or  objective  of  each  presen- 
tation.   Contact  with  a  presentation 
was  recorded  only  when  a  visitor 
stopped  there  for  at  least  the 
minimum  learning  time . 

As  a  basis  for  ranking  frequencies 
of  exposure,  a  contact  factor  was 
computed  for  each  presentation  as 
follows : 

Contact  _  number  of  contacts  observed m 
factor      number  of  days  presentation 
sampled 

HOURS  OF  ATTENDANCE 

Soon  after  the  Energy  Special 
opened,  it  became  obvious  that  the 
work  schedules  of  demonstration  per- 
sonnel needed  better  coordination 
with  peak  hours  of  visitation.  An 
attendance  survey  was  therefore  made 
by  a  volunteer  working  from  11  a.m. 
(when  the  Center  opened)  until  5  p.m. 
on  10  randomly  selected  days  between 
July  20  and  August  2.     Although  the 
center  remained  open  until  8  p.m., 
few  visitors  arrived  after  5  p.m. 
This  permitted  identification  of 
peak  periods . 

Visitation  data  for  the  entire 
Pacific  Science  Center  were  routinely 
taken  at  the  Center's  ticket  booth, 


so  the  survey  also  showed  the  percentage 
of  visitors  reaching  the  Energy  Special. 

VISITOR  CHARACTERISTICS 

To  determine  who  visits  the  Pacific 
Science  Center,  a  brief  questionnaire 
was  given  to  one  member  of  each  group 
entering  the  Center  between  August  3  and 
August  17.     The  824  questionnaires  dis- 
tributed represented  2,615  visitors. 
All  questionnaires  were  completed  on  the 
spot  and  dropped  into  a  slotted  box. 
Questions  asked  for  a  breakdown  of 
each  group  by  age  classes  and  sex,  the 
place  of  residence  for  each  member  in 
the  group,  and  the  number  of  times  the 
person  completing  the  questionnaire  had 
visited  the  Pacific  Science  Center. 

FUTURE  "SPECIALS"  QUESTIONNAIRE 

To  examine  visitor  interest  in 
selected  themes  for  future  summer 
specials,  a  brief  questionnaire  was 
used  for  approximately  a  week  in  late 
August.     This  was  provided  in  a  rack 
attached  to  the  visitor  suggestion 
box  in  the  reading  area.  Visitors 
could  choose  among  four  alternatives 
or  write  in  their  own.     The  four 
alternatives  were:     (1)  man's  five 
senses,   (2)  computers,   (3)  expansion 
of  the  Energy  Special,  and  (4)  man's 
effect  on  the  environment. 

Results  and  Discussion 

A  key  finding  of  this  study  is 
the  importance  of  feedback  early  in 
the  development  of  a  presentation 
before  creative  people  have  invested 
great  amounts  of  time,  energy,  and  ego. 
During  the  early  stages  of  development, 
promptness  of  feedback  and  ease  of 
obtaining  it  are  probably  more  important 
than  its  precision. 

The  importance  of  timely  feedback 
became  obvious  during  the  Energy  Special. 
The  staff  of  the  Pacific  Science  Center 
actively  sought  and  welcomed  feedback  as 
a  basis  for  changing  and  improving  the 
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presentations.    However,  evaluation 
did  not  begin  until  the  exhibition 
was  ready  to  open.     This  made  any 
substantial  changes  quite  painful 
for  individuals  who  had  labored  under 
pressure  for  over  a  month  to  give 
physical  form  to  their  plans  and  ideas. 
Also,  it  took  time  to  develop  replace- 
ments for  presentations  to  be  dis- 
continued.   As  a  result,  some 
ineffective  presentations  were 
retained  longer  than  desirable. 

The  evaluation  techniques  tested 
are  rated  in  table  2  and  discussed 
below  in  terms  of  speed,  burden  on 
personnel,  other  costs,  and  usefulness. 


as  good  as  the  insights  of  the  outsiders 
used,  careful  thought  should  be  given 
to  their  selection  and  preparation. 
Experienced  interpreters  and  designers 
may  bring  preconceptions  as  well  as 
special  insights  to  the  task.  Thus, 
specialists  whose  views  may  closely 
parallel  those  of  the  people  who  created 
a  presentation  may  give  a  less  accurate 
estimate  of  public  reaction  than  that 
provided  by  intelligent  people  who  are 
not  closely  associated  with  interpreta- 
tion and  have  no  specialized  knowledge 
of  the  subject  matter  being  presented. 
To  focus  the  attention  of  outsiders  on 
pertinent  matters,  a  checklist  such 
as  that  suggested  in  figure  1  can  be 
used. 


Table  2— Rating  of  six  techniques  for  evaluating  interpretive  presentations 


Technique 

Speed  of 
feedback 

Burden  on 
personnel 

Cost  for 
equipment  and 
materials 

Guarantee 
against 
bias 

Usefulness  and  limitations 

Opinion  from  panel  of 
selected  outsiders 

Excellent 

Small 

Low 

Low 

Identifies  major  problems 
before  public  presentation 

Volunteered  comments 
(via  suggestion  box) 

Good 

Small 

Low 

Low 

Can  identify  range  of  reactions; 
respondents  self-selected 

Observed  audience 
attention 

Good 

Moderate 

Low 

Good 

Requires  training.  Assumes 
that  "attention"  indicates 
effectiveness.  Respondent 
characteristics  may  differ 
at  different  presentations, 
making  comparisons  risky 

Time- lapse 
photography 

Good 

Small 

Moderate 
to  high 

Good 

Records  continually,  identifies 
use  patterns,  and  captures 
infrequent  occurrences  with 
little  burden  on  personnel. 
Area  covered  from  one  camera 
position  usually  quite  limited 

Visitor  voting  at 
individual 
presentations 

Fair 

Moderate 
to  great 

Moderate 

Moderate 

Respondent  characteristics  may 
differ  at  different  presenta- 
tions, making  comparisons 
risky 

Following  sample 
visitors 

Good  to 
fair 

Great 

Low 

Good 

Best  for  studying  visitor 
orientation  and  movements. 

Inefficient  for  rating 
visitor  interest  in  specific 
presentations 


PANEL  OF  OUTSIDERS 

One  of  the  fastest  ways  to  evaluate 
a  presentation  is  to  have  several 
thoughtful  outsiders  examine  it  and 
comment  on  its  strengths,  it  shortcom- 
ings, and  opportunities  for  improve- 
ment.   Such  evaluation  does  not  depend 
on  public  reaction  and  can  therefore 
be  used  before  a  presentation  is 
nearing  its  final  form. 

Because  outside  opinion  is  only 


VOLUNTEERED  COMMENTS 

Collection  of  volunteered  comments 
can  start  as  soon  as  a  presentation  is 
offered  to  the  public.     Such  comments 
are  easily  obtained  by  providing  a  sign 
and  suggestion  box.    They  offer  an 
evaluation  opportunity  too  often 
neglected.     Because  suggestion  boxes 
assure  anonymity  and  prevent  visitors 
from  seeing  other  comments,  they 
should  yield  better  information  than  a 
visitor  register  with  a  "comments"  column 
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CHECKLIST 

1. 

OBJECTIVES: 

a. 

From  your  observation  of  the  interpretation,  what  do  its  objectives  seem  to  be? 

b. 

Are  they  reasonable? 

c. 

(LATER)  Are  these  the  objectives  outlined  by  the  creators  of  the  interpretation? 

d. 

If  not,  why  the  discrepancy? 

2. 

AUDIENCE: 

a. 

In  this  setting,  what  are  the  likely  objectives  of  the  audience,  and  are  the 

objectives  of  the  interpretation  compatible  with  the  objectives  of  the  audience 

or  potential  audience? 

b. 

What  proportion  of  the  potential  audience  is  stopping? 

c. 

How  long  would  it  take  the  average  visitor  to  fully  experience  this  interpretation? 

d. 

How  long  are  visitors  actually  spending  with  this  interpretation? 

e. 

Do  visitors  seem  interested  or  uninterested?  Why? 

f. 

Which  age  groups  seem  interested  and  which  uninterested? 

3. 

SETTING  AND  DESIGN: 

a. 

Is  it  easy  for  visitors  to  reach  or  find  this  interpretation? 

b. 

Is  the  visitor  given  sufficient  clues  to  experience  different  elements  or  units  of  the 

interpretation  in  a  meaningful  sequence? 

c. 

Is  it  easy  and  comfortable  for  the  visitor  to  experience  this  interpretation? 

(Seating,  if  appropriate;  suitable  viewing  available  to  children;  etc.) 

4. 

CONTENT  AND  DESIGN: 

a. 

Is  any  of  the  information  incorrect?  Unclear?  Inappropriate? 

b. 

Do  any  conflicts  occur  within  this  interpretation  or  with  nearby  interpretation? 

(Consider  competition  for  attention  as  well  as  conflicts  in  subject  matter.) 

c. 

What  opportunities  for  improvement  are  available? 

d. 

Why  or  how  would  these  work  better? 

Figure  1. — Checklist  for  judging  interpretive  presentations . 


Volunteered  comments  do  not  give 
unbiased  estimates  of  average  opinion 
and  may  overrepresent  highly  motivated 
or  irritated  visitors .     However,  because 
different  visitors  respond  quite 
differently  to  any  one  presentation, 
interpreters  must  consider  the  range 
and  distribution  of  opinion  rather 
than  average  opinion. 

Volunteered  comments  tend  to 
focus  on  whatever  strikes  visitors 
as  especially  good  or  bad,  thus 
detecting  the  extremes  that  define 
the  range  of  opinion.     If  the  con- 
ditions under  which  comments  are 


collected  do  not  change,  volunteered 
comments  can  also  identify  trends. 2/ 


—    If  such  conditions  as  the  location 
of  a  comment  box  are  changed,  what  appears 
to  be  a  shift  in  visitor  opinion  might 
well  result  from  the  fact  that  comment 
boxes  in  different  locations  sample  popula- 
tions with  somewhat  different  compositions. 
Likewise,  if  different  kinds  of  visitors 
frequent  a  facility  at  different  times  of 
year,  a  change  thought  to  result  from 
improvements  in  the  facility  might  in 
actuality  result  because  an  increased 
proportion,  of  the  visitors  had  strong 
interest  in  the  things  presented. 


Table  3--Simnary  of  visitor  comments  on  exhibits  at  the  Pacific  Science 
Center's  summer  special  on  energy j  1973 


Visitor  comments 

June  28  to  July  16 

July  19  to  August  15 

Number 

Percent 

Number 

Percent 

Compl imentary 

140 

54.5 

157 

75.8 

Suggested  additions  or  improvements 

38 

14.8 

18 

8.7 

Information  poorly  communicated 

22 

8.6 

10 

4.8 

Boring 

9 

3.5 

8 

3.9 

Expand  and  improve 

10 

3.9 

7 

3.4 

Too  elementary 

9 

3.5 

5 

2.4 

Need  more  demonstrators 

24 

9.3 

2 

1.0 

Demonstrators  lack  knowledge 

5 

1.9 

0 

0 

Total 

257 

100.0 

207 

100.0 

Table  3  summarizes  visitor  com- 
ments volunteered  during  two  periods. 
Of  the  464  comments  received,  well 
over  half  were  complimentary,  which, 
by  itself,  means  little.    A  "con- 
geniality bias"  results  from  people's 
widespread  tendency  to  be  agreeable. 
The  shift  from  54.5-percent  to 
75.8-percent  complimentary  remarks 
is  not  readily  explained  by  shifts  in 
visitor  composition  or  changed  pro- 
cedures for  eliciting  comments.  This 
parallels  other  evidence  of  substan- 
tial improvement  during  the  Energy 
Special . 

Although  compliments  are  nice, 
specific  comments  and  suggestions 
are  more  likely  to  be  genuine  and 
provide  a  more  useful  basis  for 
improvement .    Some  comments  focused 
on  lack  of  demonstration  personnel, 
limitations  in  their  knowledge,  and 
level  and  effectiveness  of  communica- 
tion.    The  same  problems  were  identi- 
fied by  other  evaluation  techniques 
and  were  largely  corrected  part  way 
through  the  summer,  as  reflected  in 
the  July  19  to  August  15  columns  of 
table  3. 

OBSERVED  AUDIENCE  ATTENTION 

Observation  records  can  quickly 
identify  changes  in  audience  attention 
within  a  presentation  (fig.  2)  as  well 


4         8        12       16        20  24 
TIME  IN  MINUTES 

Figure  2 .--Observed  attention  to  the 
"candy  ecology"  laboratory  presenta- 
tion, Pacific  Science  Center,  Seattle, 
Washington,  1973.     Energy  flows  among 
plants,  grazers,  and  carnivores  were 
simulated  by  exchanges  of  candy  among 
participating  visitors . 
+  =  Observer  1.     o  =  Observer  2. 
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as  differences  between  presentations. 
Investing  a  little  time  in  training 
observers  pays  off  in  more  reliable 
results.     As  shown  in  figure  2,  two 
trained  observers  consistently 
recorded  similar  patterns  of  attention. 

Used  with  an  observer's  remarks, 
a  tape  recording  of  a  presentation 
permits  reasons  for  fluctuations  in 
attention  to  be  studied. 

TIME-LAPSE  PHOTOGRAPHY 

Time- lapse  photography  quickly 
identified  important  patterns  of 
visitor  use  during  the  Energy  Special. 
For  example,  a  time-lapse  sequence 
showed  the  adding  machine  at  the 
"family  energy  consumption"  exhibit 
to  be  in  almost  constant  use.  It 
also  showed  that  the  demonstration 
personnel  on  duty  in  the  trial  exhibit 
area  were  often  either  absent  or  too 
busy  to  interact  with  all  visitors. 
(Volunteered  comments  also  identified 
the  unavailability  of  demonstration 
personnel,  table  3.) 

For  approximately  $180  each, 
completely  self-contained  super-8 
"surveillance  cameras"  are  now  com- 
mercially available.     These  time- lapse 
systems  can  be  set  up  to  record  at 
intervals  ranging  from  1-1/4  to  90 
seconds . 

The  4- day  test  of  time- lapse 
photography  provided  several  insights 
for  planning  future  time -lapse  work. 
The  preferred  interval  between  expo- 
sures became  4  seconds.     With  intervals 
of  8  or  more  seconds,  it  was  difficult 
to  visualize  the  continuity  of  visitor 
movements  while  reviewing  the  film. 
At  4-second  intervals,  a  50-foot 
super-8  film  magazine  lasted  4  hours. 
The  high-speed  color  film  [ASA  160) 
did  not  require  supplemental  lighting; 
and  "overnight"  processing  service 
was  available  for  this  film  in  Seattle. 
The  lens  was  used  at  its  shortest  focal 
length  (8  mm)  during  the  entire  test. 
A  zoom  lens  was  not  necessary.  Review 
of  the  film  would  have  been  facilitated 


with  a  projector  equipped  to  run  at  two 
to  four  frames  per  second  and  to  stop 
on  selected  frames.     (Such  projection 
equipment  may  cost  from  $300  to  $700. 
But  a  good  film  editor  is  nearly  as 
satisfactory. ) 

Time-lapse  photography  has  four 
advantages:     (1)     It  records  continuously 
with  little  burden  on  personnel.     (2)  It 
creates  a  permanent  record  that  can  be 
studied  and  used  in  training.     (3)  By 
"compressing"  time,  it  captures  patterns 
of  use  that  are  otherwise  difficult  to 
visualize.     (4)     It  makes  a  great  amount 
of  information  readily  accessible  to  top 
managers  by  condensing  it  to  a  few  minutes 
of  viewing. 

VISITOR  VOTING 

The  voting  procedure  identified 
general  improvement  in  presentations 
during  the  summer  but  had  several 
weaknesses.    Although  more  than  half 
the  people  who  received  voting  packets 
voted  at  least  once,  only  one  person  in 
seven  voted  on  all  six  presentations. 
Visitors  (as  well  as  the  panel  of  outsiders' 
found  voting  burdensome.    Because  tearing 
a  slip  from  a  card  and  dropping  it  in  a 
multislotted  box  seemed  simple,  difficulties 
and  discomfort  with  the  voting  system  were 
quite  unexpected.     Problems  may  have 
resulted  from  a  combination  of  (1)  having 
to  carry  a  voting  packet,   (2)  being  asked 
to  concentrate  on  and  respond  to  only  one 
abstract  dimension  of  a  presentation,  and 
(3)  being  expected  to  see  the  exhibition 
more  thoroughly  than  planned.  Visitors 
seemed  generally  unable  or  unwilling  to 
partition  their  overall  impressions  in 
the  abstract  way  requested  by  the 
questions.     (Each  visitor  received  a 
single  "question  card"  designed  to  focus 
his  evaluation  on  the  enjoyability  of 
the  subject  matter,  how  effectively  it 
was  presented,  or  the  amount  of  new 
information  gained.)     If  used,  voting 
probably  should  be  limited  to  total 
reactions  to  a  single  presentation. 

An  exhibit's  location  also  seemed 
to  influence  its  rating,  with  the  highest 
ratings  given  to  the  last  exhibits 
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encountered  on  the  route  usually  taken 
by  visitors.     The  most  highly  motivated 
(and  therefore  most  enthusiastic) 
visitors  probably  saw  and  voted  at 
more  presentations  than  the  average 
visitor.     In  contrast,  the  visitor  only 
mildly  interested  in  the  whole  exhibi- 
tion was  likely  to  leave  after  seeing 
and  voting  on  the  first  one  or  two 
presentations  encountered.     Thus,  much 
of  the  vote  for  some  presentations  may 
have  come  primarily  from  the  more 
enthusiastic  visitors.     Bias  from  this 
source  is  inherent  in  the  voting  pro- 
cedures and  would  be  difficult  to  avoid. 

As  a  final  weakness,  sorting  and 
counting  the  voting  slips  required  a 
great  amount  of  time,  preventing  prompt 
feedback  to  those  creating  and  modifying 
the  exhibition. 

FOLLOWING  SAMPLE  VISITORS 

Although  direct  observation  of 
visitor  movements  showed  a  great  deal 
about  the  ways  people  move  through  a 


visitor  center,  it  was  not  an  efficient 
way  to  evaluate  specific  presentations. 
Approximately  72  man-hours  were  re- 
quired (4  hours  per  day  for  18  days) 
to  collect  data  on  171  visitors  making 
206  contacts  with  the  presentations 
offered.     The  relatively  few  encounters 
recorded  for  each  presentation  never- 
theless showed  a  tremendous  variation 
in  the  ability  of  different  presen- 
tations to  hold  visitors  for  long 
enough  to  have  meaningful  effects 
(table  4)  . 

On  the  average,  unmanned  presen- 
tations that  were  always  available 
to  visitors  had  higher  contact  factors 
than  personal  but  unscheduled  presen- 
tations.   Demonstration  personnel 
were  not  always  available.  Also, 
many  visitors  seemed  to  consciously 
avoid  interacting  with  demonstrators. 

Scheduled  presentations  had 
contact  factors  ranging  from  very 
good  to  very  bad.     The  5-minute  Chef 
Boy  0  Boy  presentation  was  scheduled 


Table  4 — Pattern  of  visitor  contacts  with  exhibits  at  the  Pacific 
Science  Center's  sumer  special  on  energy 3  1973 


Exhibit 


Period 
available 


Minimum  learning 
time 


Number  of  visitor 
contacts  and 
days  sampled 


Visitor 
contact 
factor 


Kaleidoscope^.,. 
Chef  Boy  0  Boy^-' 
Solar  cookerl/ 
Reading  area!/ 
Gasoline  shortage 

(slides )ll 
Halt  Wasteful 

(puppets)!/ 
Adding  machine!' 
Candy  Ecology  Lab. !/ 
Energy  storage!/ 
Issues!/ 
Pulleys!/ 
Puzzle!/ 
What's  a  Watt!/ 
Trade-offs!/ 
Nuclear  Radiation 

Lab. 2/ 
Fuel  hunt 

(balloons)!/ 
Card  game!/ 


Minutes 


7/1  to  9/1 

0 

42/18 

2.33 

7/1  to  9/1 

3 

30/18 

1.67 

7/17-23.  7/28  to  9/1 

0 

24/15 

1.60 

7/1  to  9/1 

0.5 

26/18 

1.44 

8/15  to  9/1 

1 

3/3 

1.00 

7/25  to  9/1 

3 

17/18 

.94 

8/9  to  9/1 

2 

6/7 

.86 

7/10  to  9/1 

7 

14/18 

.78 

7/1  to  9/1 

0.5 

13/18 

.72 

7/1-  to  9/1 

2 

8/18 

.44 

7/1  to  8/15 

1 

8/18 

.44 

8/6  to  9/1 

0.5 

3/8 

.38 

7/1  to  9/1 

3 

6/18 

.33 

7/1  to  9/1 

1 

5/18 

.28 

7/1  to  8/1 

3 

1/18 

.06 

8/10  to  9/1 

2 

0/6 

.00 

7/15  to  9/1 

2 

0/18 

.00 

I  =31.5 


—  Independent  of  demonstrators  and  always  available  for  visitor  use. 

2/ 

—  Performed  on  a  scheduled  basis.    Usually  available  the  following  number  of  times  each 
day:    Chef  Boy  0  Boy,  3;  Gasoline  shortage,  1;  Walt  Wasteful,  4;  Candy  Ecology  Lab.,  4;  What's 
a  Watt,  2;  Nuclear  Radiation  Lab.,  2;  Card  game,  1. 

—  Dependent  on  demonstrators;  demonstrator  assigned  continuously. 
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to  reach  visitors  during  peak  hours 
and  required  no  active  participation 
by  audience  members.     Because  the 
minitheater  had  no  walls  around  the 
seating  area,  people  could  drift  in 
or  out  unobtrusively.     In  contrast, 
presentations  in  the  laboratory  area 
tended  to  be  longer  (up  to  30  minutes 
for  the  Candy  Ecology  presentation) , 
required  active  audience  participation, 
and  were  enclosed  in  a  room  that 
inhibited  unobtrusive  entry  or  exit. 
Also,  the  room  had  windows  and  speakers, 
permitting  visitors  outside  the  room 
to  follow  any  activities  taking  place 
inside.     In  the  predominantly  "walk- 
through" setting,  people  seemed 
reluctant  to  commit  themselves  to 
anything  they  could  not  readily  leave. 
Also,  adults  tend  to  be  nervous  about 
participating  in  anything  that  might 
make  them  look  foolish  to  others. 

Ratios  between  potential,  possible, 
and  actual  amounts  of  contact  are 
interesting.     During  the  18  days  on 
which  visitor  movements  were  observed, 
an  average  of  15  presentations  was 
avail  able.  .3/    Thus,  the  maximum 
number  of  contacts  that  could  con- 
ceivably have  been  recorded  was 
2,565  (15  presentations  X  171  visitors). 
However,  the  unavailability  of  demon- 
stration personnel  and  the  "short 
circuit"  routes  taken  by  many  visitors 
reduced  the  total  number  of  contacts 
actually  possible  to  less  than  a  fifth 
of  this  (503) .     Only  206  contacts  were 
actually  made  by  the  171  visitors 
observed . 

On  the  average,  the  observed 
visitors  stayed  in  the  Energy  Special 
area  a  short  time.     However,  the 
average  time  increased  from  approxi- 
mately 7  minutes  during  the  first  18 
days  of  observation  to  17  minutes 
during  the  last  7,  reflecting  improve- 
ments in  the  exhibition.     The  17-minute 


—    Actually,  several  more  exhibits 
were  available  for  a  few  days,  but  pre- 
sentations available  for  less  than  4  days 
were  not  included. 


average  stay  compares  favorably  with  the 
postulated  minimum  learning  time  of  31.5 
minutes  for  all  presentations,  including 
those  unavailable  at  any  given  moment. 

VISITOR  CHARACTERISTICS 

Results  from  the  survey  of  visitor 
characteristics  showed  that  over  half  the 
visitors  sampled  were  visiting  the  Pacific 
Science  Center  for  the  first  time  and 
lived  outside  the  State  of  Washington. 
Over  60  percent  were  25  or  younger,  and 
the  most  common  group  size  was  two  persons. 
Although  only  the  period  August  3  to 
August  17  was  sampled,  results  probably 
are  close  to  what  would  have  been  found 
by  sampling  the  entire  July  1  to  September 
period  during  which  the  Energy  Special 
was  available.     Fewer  out-of-State  visitors 
and  families  with  school-age  children  may 
visit  the  Pacific  Science  Center  during 
June  and  late  August. 

HOURS  OF  ATTENDANCE 

Approximately  55  percent  of  the 
people  entering  the  Pacific  Science  Center 
eventually  arrived  at  the  Energy  Special 
(fig.  3).    To  reach  it,  they  normally 

 VISITORS  TO  PACIFIC  SCIENCE  CENTER 

 VISITORS  TO  ALLEN  BUILDING 

 VISITORS  TO  ENERGY  SPECIAL 

CE  12(L 


TIME 

Figure  3. — Average  attendance  per  half  hou. 
at  Pacific  Science  Center,  Allen  Buildin 
and  "Energy  Special,"  July  20  to  August. 
1973.  (Although  visitation  to  Pacific 
Science  Center  was  recorded  on  an  hourly 
basis,  data  have  been  adjusted  to  a  half- 
hourly  basis  to  make  them  comparable  witl 
data  for  Allen  Building  and  Energy  Specii 
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traversed  three  other  exhibit  halls 
with  floor  space  equivalent  to  about 
two  football  fields!     Traffic  through 
the  Energy  Special  therefore  reached 
its  higher  levels  approximately  45 
minutes  following  the  "after  lunch" 
peak  in  admissions  to  the  entire 
Center. 

In  addition  to  general  fatigue, 
lack  of  a  clear  traffic  pattern  may 
explain  why  more  Pacific  Science 
Center  visitors  did  not  attend  the 
Energy  Special.     At  the  main  entrance 
to  the  Energy  Special,  the  most  promi- 
nent sign  pointed  to  the  Balcony  Book 
and  Gift  Shop,  and  a  one-way  door  pre- 
vented visitors  from  backtracking 
directly  to  the  Energy  Special  once 
they  had  entered  the  gift  shop.  Also, 
visitors  were  provided  few  clues  to 
an  efficient  route  into  and  through 
the  Energy  Special.    Clarification  of 
this  route  and  addition  of  one  or  two 
attention-getting  exhibits  near  the 
most  commonly  used  entrance  to  the 
Energy  Special  might  have  increased 
attendance . 

FUTURE  "SPECIALS"  QUESTIONNAIRE 

Of  the  questionnaires  distributed 
on  future  "specials,"  245  were  returned 
Among  the  four  suggested  topics, 
"computers"  was  the  most  popular  with 
98  responses   (40  percent),  followed  by 
"man's  effect  on  the  environment"  with 
62  responses  (25  percent),  "man's  five 
senses"  with  43  responses   (18  percent) , 
and  "expansion  of  the  Energy  Special" 
with  22  responses  (9  percent) .  There 
were  20  write-in  responses  (8  percent) 
covering  such  diverse  topics  as  elec- 
tronic music,  sex,  undersea  explora- 
tion, insanity  and  witchcraft,  and  the 
population  explosion.     Responses  came 
only  from  people  who  chose  to  complete 
the  short  questionnaire,  and  highly 
motivated  visitors  may  have  been 
overrepresented . 

When  people's  preferences  were 
separated  by  age  groups,  a  "mirror 
image"  contrast  emerged  between 
"computers"  and  "man's  effect  on  the 


environment"  (fig.  4).    This  parallels 
other  findings  that,  between  childhood 
and  maturity,  people's  interest  tend  to 
shift  from  the  concrete  and  discrete 
toward  the  abstract  and  integrated  and 
from  concern  with  individual  things 
toward  processes  and  social  concerns. 
Computers  are  sensational  machines  with 
awesome  power,  associated  in  many  people's 
minds  with  physical  images  of  spinning 
tape  reels  and  flashing  lights.  In 
addition,  computers  are  means  rather 
than  ends.     In  contrast,  "man's  effect 
on  the  environment"  is  much  more  abstract, 
involving  social  concerns  and  goals  and 
the  integration  of  diverse  processes  and 
factors . 

90  r 


80  - 


10  - 


0I  1  1  1  1  

CHILDREN      TEENAGERS       YOUNG  ADULTS 
(5-12)  (13-18)  ADULTS  (25+) 

(19-25) 

AGE 

Figure  4. — Effect  of  age  on  preferred  themes 
for  future  "summer  specials"  at  the  Pacific 
Science  Center. 


Although  the  data  of  figure  4 
come  only  from  "self-selected"  visitors, 
they  nevertheless  warn  that  presentations 
aimed  at  one  age  group  may  largely  miss 
another. 
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Conclusions 

Two  groups  of  conclusions  result 
from  evaluation  of  the  Pacific  Science 
Center's  "summer  special"  exhibition, 
"Energy  and  Its  Relation  to  Man  and  the 
Environment."    One  of  these  concerns 
evaluation  techniques;  the  other,  the 
"Energy  Special"  itself. 

GUIDELINES  FOR  EVALUATION 

From  experience  gained  with  six 
evaluation  techniques,  12  guidelines 
for  evaluation  are  offered: 

1.  When  an  interpretive  presenta- 
tion is  being  developed,  the  speed  of 
evaluation  is  generally  more  important 
than  its  precision. 

2.  Comments  from  a  few  thoughtful 
outsiders  can  identify  major  problems 
at  an  early  stage  before  large  invest- 
ments of  time,  money,  and  personal 
involvement  make  changes  awkward. 
Pretesting  could  often  avoid  costly 
errors  and  should  be  routine  for 

most  presentations. 

3.  Documentation  is  important, 
becoming  more  than  mere  opinion  to  be 
countered  by  the  opinion  of  anyone 
reluctant  to  change.     Graphs,  tables, 
and  other  summaries  can  separate 
widely  shared  reactions  from  those 

of  isolated  "cranks"  and  can  uncover 
specific  misunderstandings  and  problems. 

4.  Evaluation  need  not  be  com- 
plicated or  expensive. 

5.  Once  a  presentation  is  open 
to  the  public,  suggestion  boxes  will 
elicit  many  useful  comments.  Although 
such  volunteered  comments  usually 
include  many  gratuitous  compliments 
and  define  no  "unbiased  average," 
they  do  identify  the  range  of  visitor 
reactions  as  well  as  specific  problems 
or  opportunities  for  improvement. 

6.  When  collected  in  a  consistent 
manner,  volunteered  comments  obtained 
at  different  times  can  identify  trends 
in  effectiveness. 


7.  Time- lapse  photography  and 
direct  observation  of  audience  attention 
can  identify  major  difficulties  in  a  pre- 
sentation rather  quickly  and  at  reason- 
able costs  in  manpower  and  money. 

8.  Following  randomly  selected 
visitors  can  provide  useful  insights 
into  the  ways  visitors  perceive  and  find 
their  way  among  the  components  of  a 
presentation.     However,  the  technique 

is  time  consuming  and,  in  focusing  on 
one  visitor  at  a  time,  is  not  an  efficient 
way  to  examine  use  levels  or  patterns  at 
specific  points  within  a  building  or 
exhibit  room. 

9.  Because  different  people  will 
respond  to  presentations  quite  differ- 
ently, responses  in  some  cases  need  to 
be  classified  by  the  age,  sex,  and 
perhaps  other  observable  characteristics 
of  visitors. 

10.  Direct  comparisons  of  visitor 
response  to  different  exhibits  or  pre- 
sentations may  be  misleading.  Because 
of  inadequate  orientation  as  well  as 
differences  in  time  and  motivation,  the 
kinds  of  visitors  predominating  at  dif- 
ferent presentations  may  differ  markedly. 

11.  Visitors  should  not  be  expected 
to  undertake  complex  or  lengthy  evaluation 
tasks.    Because  visiting  interpretive 
presentations  is  usually  a  leisure  time 
activity,  carrying  evaluation  forms  or 
reading  even  slightly  complex  instructions 
may  be  perceived  as  a  substantial  burden. 

12.  Requests  that  people  respond  to 
only  one  abstract  dimension  of  a  presen- 
tation are  not  likely  to  be  understood. 

THE  ENERGY  SPECIAL 

Three  different  measures  indicated 
improvement  during  the  Energy  Special. 
Favorable  comments  in  the  suggestion  box, 
favorable  "votes,"  and  average  length  of 
stay  all  increased.  Presentations 
probably  would  have  improved  in  any  case 
as  a  result  of  increased  experience  among 
personnel,  informal  feedback,  and  guidance 
from  supervisors.     However,  formal  evalua- 
tion accelerated  improvement  by  diagnosing 
and  documenting  specific  problems.  These 
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included  a  mismatch  between  peak  hours 
of  visitation  and  the  availability  of 
demonstration  personnel,  the  distraction 
a  visitor-operated  exhibit  caused  for 
a  nearby  presentation,  and  presentations 
that  generated  little  visitor  interest. 

Evaluation  identified  three  addi- 
tional difficulties: 

1.  Having  personnel  available  to 
answer  questions  and  discuss  exhibits 
was  not  very  efficient,  yielding  few 
visitor  contacts  for  the  manpower 
expended.    Similar  results  are  likely 
whenever  the  flow  of  visitors  is 
highly  uneven--at  times  overtaxing 
the  ability  of  personnel  to  interact 
with  interested  visitors,  at  other 
times  leaving  such  personnel  nearly 
idle. 

2.  Visitation  to  and  enthusiasm 
at  the  Energy  Special  might  have  been 
substantially  greater  if  it  had  been 
easier  to  reach.    Of  all  visitors  to 
the  Pacific  Science  Center,  55  percent 
did  reach  the  Energy  Special,  but 
usually  after  passing  through  three 
other  buildings  of  exhibits. 

In  addition,  eye-catching  exhibits 
near  the  entrance  to  the  Energy  Special 
might  have  captured  the  interest  of 
more  visitors,  a  parallel  to  the 
"western"  writer's  recommendation, 
"Shoot  the  sheriff  on  page  1." 

3.  The  enclosed  "laboratory 
area"  provided  substantially  fewer 
visitor  contacts  than  the  "minitheater," 
where  visitors  could  drift  in  or  out. 
Many  visitors,  especially  adults,  tend 
to  avoid  "classroom"  settings  where 
escape  is  difficult,  participation  is 
required,  and  ignorance  may  be  exposed. 
However,  people  >who  did  visit  the 
laboratory  area  rated  it  quite  highly. 

Evaluation  of  interpretive  pres- 
entations probably  will  continue  to  be 
more  art  than  science.    A  great  amount 
of  somewhat  imprecise  information  can 
be  obtained  at  little  cost  in  time  and 
money  and  with  little  burden  on  visitors. 
With  some  understanding  of  the  sources 


of  bias  and  with  overlapping  data 
collection  procedures  as  a  check  on  any 
one  method,  managers  of  interpretive 
programs  should  be  able  to  avoid  the 
major  pitfalls  of  using  data  from  non- 
representative  samples  of  their  visitors. 
Many  problems  can  be  diagnosed  from 
quite  limited  information.  Nevertheless, 
managers  must  still  weigh  the  risks  of 
somewhat  biased  information  against  the 
costs  of  increased  precision. 

****** 
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The  mission  of  the  PACIFIC  NORTHWEST  FOREST 
AND  RANGE  EXPERIMENT  STATION  is  to  provide  the 
knowledge,  technology,  and  alternatives  for  present  and 
future  protection,  management,  and  use  of  forest,  range,  and 
related  environments. 

Within  this  overall  mission,  the  Station  conducts  and 
stimulates  research  to  facilitate  and  to  accelerate  progress 
toward  the  following  goals: 

1.  Providing  safe  and  efficient  technology  for  inventory, 
protection,  and  use  of  resources. 

2.  Developing  and  evaluating  alternative  methods  and 
levels  of  resource  management. 

3.  Achieving  optimum  sustained  resource  productivity 
consistent  with  maintaining  a  high  quality  forest 
environment. 

The  area  of  research  encompasses  Oregon,  Washington, 
Alaska,  and,  in  some  cases,  California,  Hawaii,  the  Western 
States,  and  the  Nation.  Results  of  the  research  are  made 
available  promptly.  Project  headquarters  are  at: 

Fairbanks,  Alaska  Portland,  Oregon 

Juneau,  Alaska  Olympia,  Washington 

Bend,  Oregon  Seattle,  Washington 

Corvallis,  Oregon  Wenatchee,  Washington 
La  Grande,  Oregon 


Mailing  address:   Pacific  Northwest  Forest  and  Range 
Experiment  Station 
P.O.  Box  3141 
Portland,  Oregon  97208 


GPO  997-387 
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